Engineers use compression (also called source coding or source encoding) to reduce the bit rate of digital video. Compression decreases the cost of storing and transmitting video information by converting the information into a lower bit rate form. Decompression (also called decoding) reconstructs a version of the original information from the compressed form. A “codec” is an encoder/decoder system.
Over the last two decades, various video codec standards have been adopted, including the H.261, H.262 (MPEG-2 or ISO/IEC 13818-2), H.263 and H.264 (AVC or ISO/IEC 14496-10) standards and the MPEG-1 (ISO/IEC 11172-2), MPEG-4 Visual (ISO/IEC 14496-2) and SMPTE 421M standards. More recently, the HEVC standard is under development. A video codec standard typically defines options for the syntax of an encoded video bitstream, detailing parameters in the bitstream when particular features are used in encoding and decoding. In many cases, a video codec standard also provides details about the decoding operations a decoder should perform to achieve correct results in decoding.
A basic goal of compression is to provide good rate-distortion performance. So, for a particular bit rate, an encoder attempts to provide the highest quality of video. Or, for a particular level of quality/fidelity to the original video, an encoder attempts to provide the lowest bit rate encoded video. In practice, depending on the use scenario, considerations such as encoding time, encoding complexity, encoding resources, decoding time, decoding complexity, decoding resources, overall delay, and/or smoothness in playback also affect decisions made during encoding and decoding.
For example, consider use scenarios such as video playback from storage, video playback from encoded data streamed over a network connection, and video transcoding (from one bit rate to another bit rate, or one standard to another standard). At the encoder side, such applications may permit off-line encoding that is not at all time-sensitive. Therefore, an encoder can increase encoding time and increase resources used during encoding to find the most efficient way to compress video, and thereby improve rate-distortion performance. If a small amount of delay is also acceptable at the decoder side, the encoder can further improve rate-distortion performance, e.g., by exploiting inter-picture dependencies from pictures farther ahead in a sequence.
On the other hand, consider use scenarios such as remote desktop conferencing, surveillance video, video telephony and other real-time communication scenarios. Such applications are time-sensitive. Low latency between recording of input pictures and playback of output pictures is a key factor in performance. When encoding/decoding tools adapted for non-real-time communication are applied in real-time communication scenarios, overall latency is often unacceptably high. The delays that these tools introduce during encoding and decoding may improve performance for regular video playback, but they disrupt real-time communication.